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C-36 


Briefing  Agenda 


The  One  Big  Thing 

What’s  going  on  now 

►  Shift  the  paradigm 

The  next  hurdles 

►  How  do  we  communicate  this  is 
right  thing  to  do? 

►  Project-level  process  workflows 

What  I  Need  From  You 
My  Crystal  Ball 


the 


The  One  Big  Thing 


■  Not  leaving  the 
easy  stuff: 

►  a  105mm  @  a 
meter 

►  Ora  37mm  @ 
0.3m 


What  this  means  to  me: 

-How  do  we  know  we  have  good  data? 
-What  do  good  decisions  “look”  like? 
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BUILDING  STRONG© 


Why  is  Everything  Upside  Down? 


mag&flag 

•  Easy  to  understand  (one  switch  &  only  5  settings) 

•  Belief  that  “everything”  gets  picked  up 

•  Been  doing  this  since  the  1940’s 

Current  DGM 

•  Those  geophysicists  are  weird  and  speak  a  different 
language  (?Geophysees?) 

•  But  their  tools  seem  work  pretty  well 

•  Been  doing  this  since  mid-1990’s 

Advanced  Classification 

•  Its  new  &  I  don’t  understand  it 

•  What  the  heck  are  Eigenvalues... &  I  don’t  understand  it 

•  What  the  heck  is  a  feature  vector... &  I  don’t  understand  it 

•  These  geophysicists  are  more  weird  than  those  others  &  I 
think  they’re  using  English  words  but  I’m  not  sure 


Implicit  Level 
of  “Trust” 

Most 


Least 


1  I 


© 
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BUILDING  STRONG, 


Why  is  Everything  Upside  Down? 


Implicit  Level 
of  “Comfort” 


mag&flag 

•  Easy  to  understand  (one  switch  &  only  5  settings) 

•  Belief  that  “everything”  gets  picked  up 

•  Been  doing  this  since  the  1940’s 

Current  DGM 

•  Those  geophysicists  are  weird  and  speak  a  different 
language  (?Geophysees?) 

•  But  their  tools  seem  work  pretty  well 

•  Been  doing  this  since  mid-1990’s 

Advanced  Classification 

•  Its  new  &  I  don’t  understand  it 

•  What  the  heck  are  Eigenvalues... &  I  don’t  understand  it 

•  What  the  heck  is  a  feature  vector... &  I  don’t  understand  it 

•  These  geophysicists  are  more  weird  than  those  others  &  I 
think  they’re  using  English  words  but  I’m  not  sure 


1  l 
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BUILDING  STRONG, 


Need  to  Change  The  Paradigm 


What’s  really  going  on 

we’re  leaving  behind 


"  mag&flag 

•  Least  sensitive  instrumentation 

•  Most  prone  to  poor  system  performance 

•  Cannot  quantify  what  a  “dig  decision”  means 


■  Current  DGM 

•  Very  sensitive  instrumentation 

•  System  performance  can  be  monitored  throughout 

•  Dig  decisions  boil  down  to  “Big”  vs.  “Small” 

■  Advanced  Classification 

•  Most  sensitive  instrumentation 

•  System  performance  is  quantified 

•  Dig  decisions  built  from  the  underlying  physics 
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BUILDING  STRONG, 


How  Do  We  Communicate  This  Is  The  Right 

Thing  To  Do? 

What’s  really  going  on  The  New  paradigm 


■  mag&flag 

•  Least  sensitive  instrumentation 

•  Most  prone  to  poor  system  performance 

•  Cannot  quantify  what  a  “dig  decision”  means 


■  Current  DGM 

•  Very  sensitive  instrumentation 

•  System  performance  can  be  monitored  throughout 

•  Dig  decisions  boil  down  to  “Big”  vs.  “Small” 

■  Advanced  Classification 

•  Most  sensitive  instrumentation 

•  System  performance  is  quantified 

•  Dig  decisions  built  from  the  underlying  physics 


Individual  blades  of  grass 
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BUILDING  STRONG, 


How  Do  We  Communicate  This 
Is  The  Right  Thing  To  Do? 


“That’s  the  way  we’ve  always  done  it” 
=  noise  to  us 


100 


0  . 

0  200  400  600  800  1000 

Number  of  Clutter  Items 


“Look  at  that  pretty  ROC  curve” 
=  noise  to  them 


Boils  down  to  choice 


■  The  right  people 

■  The  right  tools 

■  A  plan  that  makes  sense 
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What  does  “A  plan  that  makes  sense” 

mean? 


■  The  ROC  curve 
needs  to  have 
meaning 

►  Why  did  you  stop 
digging 

►  Why  did  you  select 
the  things  you  did  dig 

►  “Comfort”  needs  to 
come  out  of  the 
equation 


p 

1  clutter 
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BUILDING  STRONG, 


What  does  “A  plan  that  makes  sense” 

mean? 


Camp  Butner: 
Artillery  Impact 
Area  (Area  A) 


37-mm  projectiles 
105-mm  projectiles 
155-mm  projectiles 


The  Classification  Process  needs  to  make  sense  for  the 
whole  project  site 

(aka  Munitions  Response  Site) 
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BUILDING  STRONG, 


Process  Workflow 


E 

O) 

c 


o 


Z) 


This  is  an  ESTCP 
demonstration 


LO 

O 


699,300  b29,?5G  699,400 


© 


UTM  Easting  (m) 


BUILDING  STRONG, 


Need  A  Process  Workflow  For  The  Whole  MRS 

This  could  look  like  an 


BUILDING  STRONG, 


n=750 


Blind  seed  ISO 


Blind  seed  inert  items 
if  possible  (any 
number  will  benefit) 


Select  sub-area  having  n 
anomalies 


What  an  Apply 
Globally  scenario 
really  means 


;  Define  ; 

■  Classification 


-Test/Valldate  ^ 
;  ;  process  ; 


Adjust 

T 


: :  Apply; : 
Globally 


Stop 


n=250 


Yes 


Perform 
classification 
processes  according 
to  plan 


No 


Reveal  all 
ground  truth 
and  analyze 


Apply  classification 
process  to  all 
remaining  anomalies 


■» 

■» 

■» 


Assess  QC 


A 

No 


New  blind  test 

■ 

needed? 

Adjust  Procedures  to 
achieve  requirements 


Yes 


Can  requirements  be 
achieved 


<■ 


No 


Does  process 
perform  to 
requirements? 


Assess  QC  (e.g.  would  improved  data 
quality  improve  successes? 


Assess  Processes  (e.g.  would  improved 
decision  boundaries  improve  successes? 


Assess  Other  (e.g.  complex  clutter,  low 
SNR? 


Yes 


Apply  adjustments  to 
procedures  or  decision 
boundaries,  as  appropriate, 
to  improve  performance 


Assess  Processes 


Assess  Findings 


Additional  digs  as 
required 


Done 


in  ii 
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BUILDING  STRONG, 


What  I  Need  From  You 


■  How  do  I  explain  we  didn’t  miss  the  easy  stuff? 

■  What  are  the  important  “back-up”  materials  I  need  to 
show  we  made  the  right  decisions? 

■  Classification  Processes  designed  for  the  whole  MRS 

■  Challenge  people  who  think  they  “get  it  all”  using  lesser 
technologies 

■  Up  for  discussion: 


►  What  are  reasonable  and  achievable  goals  for 

•  %  UXO  recovered 

•  Clutter  not  needing  to  be  excavated 


BUILDING  STRONG, 


My  Magic  8  Ball 


term: 

Will  ©feiofetlffiQiQKa^r^la^ffe  now? 
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BUILDING  STRONG 


My  Magic  8  Ball 


Long  term: 

■  Will  we  ^pt^eribfeteaiR^iidiiig-fD^KShd  evacuations  based  on  munitions  ID? 
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Thank  You 


Army  UXO  Safety  Program 


Learn  and  Follow 
the 


RETREAT:  Do  not  disturb,  touch  or  move  it 
Do  nor  give  or  throw  it  awoy 

REPORT:  Call  911 
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